{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fc147041",
   "metadata": {},
   "source": [
    "# Comparison between kernel-based training and variational circuit using Covalent and Pennylane\n",
    "\n",
    "\n",
    "In this tutorial, we solve a classification problem using kernel-based training variational training and compare the number of quantum circuits that are evaluated in each case. This is an important parameter in the NISQ era as each circuit evaluation on a real quantum hardware is expensive and time consuming. It is observed that kernel-based training requie only a fraction of the number of circuit evaluations required by variational based training. We show how Covalent framework can help in these kind of situations where multiple evaluations of quantum circuits are needed.\n",
    "\n",
    "This demo is based on the Pennylane tutorial [**Kernel-based training of quantum models with scikit-learn**](https://pennylane.ai/qml/demos/tutorial_kernel_based_training.html).\n",
    "\n",
    "We start by importing all the necessary libraries used in the tutorial.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ab369db7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "covalent\n",
      "matplotlib==3.5.1\n",
      "numpy==1.24.2\n",
      "pennylane==0.25.1\n",
      "pennylane-sf==0.20.1\n",
      "scikit-learn==1.0.2\n",
      "torch==2.0.0\n",
      "torchvision==0.15.1\n"
     ]
    }
   ],
   "source": [
    "with open(\"./requirements.txt\", \"r\") as file:\n",
    "    for line in file:\n",
    "        print(line.rstrip())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1e61d293",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install necessary packages\n",
    "# !pip install -r ./requirements.txt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a75f7fac",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "from torch.nn.functional import relu\n",
    "\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "import pennylane as qml\n",
    "from pennylane.templates import AngleEmbedding, StronglyEntanglingLayers\n",
    "from pennylane.operation import Tensor\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import covalent as ct\n",
    "\n",
    "np.random.seed(42)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ee9e11b",
   "metadata": {},
   "source": [
    "# Load, preprocess and split data\n",
    "\n",
    "We will be using the Iris dataset for the classification problem in this tutorial. We do some necessary preprocessing on the data and split them into training and test dataset.\n",
    "\n",
    "### Iris dataset\n",
    "\n",
    "Iris dataset is a very well known dataset used for pattern recognition and classification. It consists of samples belonging to 3 classes namely: Iris Setosa, Iris Versicolour and Iris Virginica. Each sample has four features related to the length and width of sepal and petal. In this tutorial we will be classifying first two classes that correspond to first 100 samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cdb32772",
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = load_iris(return_X_y=True)\n",
    "\n",
    "# pick inputs and labels from the first two classes only,\n",
    "# corresponding to the first 100 samples\n",
    "X = X[:100]\n",
    "y = y[:100]\n",
    "\n",
    "# scaling the inputs is important since the embedding we use is periodic\n",
    "scaler = StandardScaler().fit(X)\n",
    "X_scaled = scaler.transform(X)\n",
    "\n",
    "# scaling the labels to -1, 1 is important for the SVM and the\n",
    "# definition of a hinge loss\n",
    "y_scaled = 2 * (y - 0.5)\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_scaled, y_scaled)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04081889",
   "metadata": {},
   "source": [
    "# Kernel-based training \n",
    "\n",
    "Suppose we use a linear classifier for solving a classification problem, it need not always be the case that the classifier is successful. This can be attributed to the fact that samples are not always linearly separable. A common trick to overcome this is to transform the data points to a *higher dimensional* space and perform a linear classification there. This works because performing a linear classification in a *higher dimensional* space corresponds to a non-linear classification in the original space.\n",
    "\n",
    "In order to perform calculations in a higher dimensional space we use a special function called the *kernel*. The beauty of these functions is that they are able to perform these computations with out transforming the sample data points to the higher dimensional space. Therefore, kernel-based training can bypass the processing and measurement parts of common variational circuits, and only depends on the data encoding. If the loss function used is the hinge loss, the kernel method corresponds to a standard support vector machine (SVM) in the sense of a maximum-margin classifier. In quantum kernel-based training, we will be using quantum computers for computing the kernel function. In this tutorial, we will be PennyLane’s kernels module for creating simple implementations of Quantum Embedding Kernels.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b20137e",
   "metadata": {},
   "source": [
    "## Constructing the workflow for kernel-based training\n",
    "\n",
    "The workflow for training based on kernel is broken down into subtasks (functions with the Covalent electron decorator) as follows.\n",
    "\n",
    "1. `kernel` - The quantum kernel evaluator which calculates the kernel function value between two values. It performs angle embedding for encoding the data features.\n",
    "2. `kernel_matrix` - Compute the matrix whose entries are the kernel evaluated on pairwise data from sets A and B.\n",
    "3. `fit_SVC` - It fits a Support Vector Classifer for the supplied kernel matrix.\n",
    "4. `predict_SVC` - It predicts the classes using the trained SVC.\n",
    "5. `get_accuracy_score` - Returns the accuracy score for the predicted classes.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c572c82d",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_qubits = len(X_train[0])  # Number of qubits needed for data encoding\n",
    "\n",
    "dev_kernel = qml.device(\n",
    "    \"default.qubit\", wires=n_qubits\n",
    ")  # Simulator used for performing kernel-based training.\n",
    "\n",
    "projector = np.zeros((2**n_qubits, 2**n_qubits))\n",
    "projector[0, 0] = 1\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "@qml.qnode(dev_kernel)\n",
    "def kernel(x1, x2):\n",
    "    AngleEmbedding(x1, wires=range(n_qubits))\n",
    "    qml.adjoint(AngleEmbedding)(x2, wires=range(n_qubits))\n",
    "    return qml.expval(qml.Hermitian(projector, wires=range(n_qubits)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e0f2e1a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "@ct.electron()\n",
    "def kernel_matrix(A, B):\n",
    "    return np.array([[kernel(a, b) for b in B] for a in A])\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def fit_SVC(X_train, y_train, kernel_matrix):\n",
    "    svm = SVC(kernel=kernel_matrix).fit(X_train, y_train)\n",
    "    return svm\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def predict_SVC(svm, X_test):\n",
    "    predictions = svm.predict(X_test)\n",
    "    return predictions\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def get_accuracy_score(predictions, y_test):\n",
    "    return accuracy_score(predictions, y_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4ef5306",
   "metadata": {},
   "source": [
    "We create a waorkflow `kernel_workflow` using Covalent lattice decorator for training a kernel-based classifier and for predicting classes of the test set. We then dispatch the workflow to the dask executor using Covalent `dispatch`. The results are later retrieved by using Covalent `get_result` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2665f5e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "@ct.lattice\n",
    "def kernel_workflow(X_train, y_train, X_test, y_test, kernel_matrix):\n",
    "    svm = fit_SVC(X_train, y_train, kernel_matrix)\n",
    "    predictions = predict_SVC(svm, X_test)\n",
    "    acc = get_accuracy_score(predictions, y_test)\n",
    "    return acc\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4b0a1a2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "dispatch_id = ct.dispatch(kernel_workflow)(X_train, y_train, X_test, y_test, kernel_matrix)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ffb39054",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kernel_result = ct.get_result(dispatch_id=dispatch_id, wait=True)\n",
    "kernel_result.result\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b8310e8",
   "metadata": {},
   "source": [
    "# Variational training\n",
    "\n",
    "Variational circuits consists of quantum circuits that have trainable parameters in them. These parameters are trained with a classical optimization algorithm which looks for better set of parameters at each step of training. The expressivity of the variational algorithm depends on the depth of the ansatz used in the variational circuit.\n",
    "\n",
    "We will be using gradient-based optimizers and we will use parameter-shift differentiation method for quantum nodes as it works on real quantum hardwares as well. Taking this approach will help us calculate the number of circuit executions needed more accurately."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dda2098",
   "metadata": {},
   "source": [
    "## Constructing the workflow for variational training\n",
    "\n",
    "The workflow for training based on variational circuit is broken down into subtasks (functions with the Covalent electron decorator) as follows.\n",
    "\n",
    "1. `quantum_model` - Evaluates the quantum variational circuit.\n",
    "2. `quantum_model_plus_bias` - Adds bias to the output of the circuit.\n",
    "3. `hinge_loss` - Calculates the hinge loss between the predictions and target value.\n",
    "4. `get_params` - Returns random initial parameters as torch objects.\n",
    "5. `get_optimizer` - Returns the the optimizer needed for training.\n",
    "6. `get_batch_data` - Returns the samples used for training in a particular batch.\n",
    "7. `step_optimizer` - Performs one step of optimization during training.\n",
    "8. `run_iteration` - Runs one complete iteration of training.\n",
    "9. `get_random` - Returns random indexes needed to choose batch.\n",
    "10. `get_torch` - Converts an array into a torch object.\n",
    "11. `get_pred` - Predict the class based on the output of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9b082a7b",
   "metadata": {},
   "outputs": [],
   "source": [
    "dev_var = qml.device(\"default.qubit\", wires=n_qubits)\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "@qml.qnode(dev_var, interface=\"torch\", diff_method=\"parameter-shift\")\n",
    "def quantum_model(x, params):\n",
    "\n",
    "    AngleEmbedding(x, wires=range(n_qubits))\n",
    "\n",
    "    StronglyEntanglingLayers(params, wires=range(n_qubits))\n",
    "    return qml.expval(qml.PauliZ(0))\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def quantum_model_plus_bias(x, params, bias):\n",
    "\n",
    "    return quantum_model(x, params) + bias\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def hinge_loss(predictions, targets):\n",
    "\n",
    "    all_ones = torch.ones_like(targets)\n",
    "    hinge_loss = all_ones - predictions * targets\n",
    "\n",
    "    hinge_loss = relu(hinge_loss)\n",
    "    return hinge_loss\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "46fb6b81",
   "metadata": {},
   "outputs": [],
   "source": [
    "@ct.electron()\n",
    "def get_params(n_layers):\n",
    "    params = np.random.random((n_layers, n_qubits, 3))\n",
    "    params_torch = torch.tensor(params, requires_grad=True)\n",
    "    bias_torch = torch.tensor(0.0)\n",
    "    return params_torch, bias_torch\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def get_optimizer(params_torch, bias_torch):\n",
    "    return torch.optim.Adam([params_torch, bias_torch], lr=0.1)\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def get_batch_data(batch_ids):\n",
    "    X_batch = X_train[batch_ids]\n",
    "    y_batch = y_train[batch_ids]\n",
    "\n",
    "    X_batch_torch = torch.tensor(X_batch, requires_grad=False)\n",
    "    y_batch_torch = torch.tensor(y_batch, requires_grad=False)\n",
    "\n",
    "    return X_batch_torch, y_batch_torch\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def step_optimizer(opt, params_torch, bias_torch, X_batch_torch, y_batch_torch):\n",
    "    def closure():\n",
    "        opt.zero_grad()\n",
    "        preds = torch.stack(\n",
    "            [quantum_model_plus_bias(x, params_torch, bias_torch) for x in X_batch_torch]\n",
    "        )\n",
    "        loss = torch.mean(hinge_loss(preds, y_batch_torch))\n",
    "\n",
    "        loss.backward()\n",
    "        return loss\n",
    "\n",
    "    opt.step(closure)\n",
    "    current_loss = closure().detach().numpy().item()\n",
    "    return opt, current_loss\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def get_random(n, batch_size):\n",
    "    return np.random.choice(n, batch_size)\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def get_torch(x):\n",
    "    return torch.tensor(x)\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def get_pred(pred_torch):\n",
    "    pred = pred_torch.detach().numpy().item()\n",
    "    if pred > 0:\n",
    "        pred = 1\n",
    "    else:\n",
    "        pred = -1\n",
    "    return pred\n",
    "\n",
    "\n",
    "@ct.electron()\n",
    "def run_iteration(opt, batch_size, params_torch, bias_torch):\n",
    "    batch_ids = get_random(len(X_train), batch_size)\n",
    "\n",
    "    X_batch_torch, y_batch_torch = get_batch_data(batch_ids=batch_ids)\n",
    "\n",
    "    opt, current_loss = step_optimizer(\n",
    "        opt=opt,\n",
    "        params_torch=params_torch,\n",
    "        bias_torch=bias_torch,\n",
    "        X_batch_torch=X_batch_torch,\n",
    "        y_batch_torch=y_batch_torch,\n",
    "    )\n",
    "\n",
    "    return opt, current_loss\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf19f730",
   "metadata": {},
   "source": [
    "## Training workflow\n",
    "\n",
    "We create two sublatices `quantum_model_train` and `quantum_model_predict` using Covalent lattice decorator for creating the workflow for training and predicting using the variational circuit.\n",
    "\n",
    "The `quantum_model_train` workflow runs each step of optimization inside the sub lattice and hence it can save the intermediate optimization steps. This has an advantage when the code is run on an actual quantum hardware as usually each step would be both time and resource costly and hence we would want to monitor not only at the main optimization level, but also at the iteration level. From the DAGs we can see how the results are saved as each step and if any step fails we can use the saved previous computed results to continue further.\n",
    "\n",
    "\n",
    "We then create the main workflow for training and predicting with `variational_circuit` Covalent lattice. We then dispatch the workflow using Covalent `dispatch`. The results are retrieved by using Covalent `get_result` method. The output is used to compute the accuracy and to plot the loss function.\n",
    "\n",
    "## Directed Acyclic Graph (DAG)\n",
    "\n",
    "In order to verify if the workflow construction is properly defined, we can use the draw method in the predict_workflow or check out the DAG in the Covalent UI. The graph contains information on execution status, task definition, runtime, input parameters, and more. \n",
    "\n",
    "The Directed Acyclic Graph (DAG) generated for `variational_circuit` workflow is shown below:\n",
    "\n",
    "![DAG](assets/covalent-kernel-DAG.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "15612d53",
   "metadata": {},
   "outputs": [],
   "source": [
    "@ct.electron()\n",
    "@ct.lattice()\n",
    "def quantum_model_train(n_layers, steps, batch_size):\n",
    "\n",
    "    params_torch, bias_torch = get_params(n_layers)\n",
    "\n",
    "    opt = get_optimizer(params_torch, bias_torch)\n",
    "\n",
    "    loss_history = []\n",
    "\n",
    "    for i in range(steps):\n",
    "\n",
    "        opt, current_loss = run_iteration(\n",
    "            opt=opt, batch_size=batch_size, params_torch=params_torch, bias_torch=bias_torch\n",
    "        )\n",
    "\n",
    "        loss_history.append(current_loss)\n",
    "\n",
    "    return params_torch, bias_torch, loss_history\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "672fcab0",
   "metadata": {},
   "outputs": [],
   "source": [
    "@ct.electron()\n",
    "@ct.lattice()\n",
    "def quantum_model_predict(X_pred, trained_params, trained_bias):\n",
    "\n",
    "    p = []\n",
    "    for x in X_pred:\n",
    "\n",
    "        x_torch = get_torch(x=x)\n",
    "        pred_torch = quantum_model_plus_bias(x=x_torch, params=trained_params, bias=trained_bias)\n",
    "\n",
    "        pred = get_pred(pred_torch=pred_torch)\n",
    "        p.append(pred)\n",
    "    return p\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a9f3ee4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "@ct.lattice()\n",
    "def variation_circuit(n_layers, steps, batch_size):\n",
    "    trained_params, trained_bias, loss_history = quantum_model_train(\n",
    "        n_layers=n_layers, steps=steps, batch_size=batch_size\n",
    "    )\n",
    "    pred_test = quantum_model_predict(\n",
    "        X_pred=X_test, trained_params=trained_params, trained_bias=trained_bias\n",
    "    )\n",
    "    return loss_history, trained_params, trained_bias, pred_test\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "c4e0b138",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy on test set: 0.16\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQiUlEQVR4nO3dfZBdd13H8feHhBYJ0BYaGUgiDRKsGR5s2Sll6kOhRZPoJA5PNmMH0A4ZHao8jU4ZmKJV/yhFUMaKRh5KUVrbgrhCpIMlDjNMW7IBWpqUwhKQJC0mlBJQRtoMX/+4J3jZ7CYbm7OX3N/7NXNn7znnt+d+f/PL7Cfn4f5OqgpJUrseMeoCJEmjZRBIUuMMAklqnEEgSY0zCCSpcQaBJDWutyBI8t4k+5LcNcf2JHlnkukkdyY5u69aJElz6/OI4BpgzRG2rwVWda9NwLt6rEWSNIfegqCqPgV86whNNgDX1sBtwKlJntRXPZKk2S0e4WcvA3YPLe/p1t03s2GSTQyOGliyZMlzzjzzzAUpUJLGxfbt279ZVUtn2zbKIJi3qtoMbAaYmJioqampEVckSSeWJP8x17ZR3jW0F1gxtLy8WydJWkCjDIJJ4OXd3UPnAgeq6rDTQpKkfvV2aijJdcD5wOlJ9gBvAR4JUFV/A2wB1gHTwPeA3+qrFknS3HoLgqraeJTtBby6r8+XJM2P3yyWpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIa12sQJFmT5J4k00kum2X7TyXZmuRzSe5Msq7PeiRJh+stCJIsAq4G1gKrgY1JVs9o9mbghqo6C7gI+Ou+6pEkza7PI4JzgOmq2lVVDwLXAxtmtCngcd37U4B7e6xHkjSLPoNgGbB7aHlPt27YHwEXJ9kDbAF+b7YdJdmUZCrJ1P79+/uoVZKaNeqLxRuBa6pqObAO+ECSw2qqqs1VNVFVE0uXLl3wIiVpnPUZBHuBFUPLy7t1wy4BbgCoqluBRwGn91iTJGmGPoNgG7AqycokJzG4GDw5o83XgQsAkvwsgyDw3I8kLaDegqCqDgKXAjcDdzO4O2hHkiuSrO+avQF4VZI7gOuAV1ZV9VWTJOlwi/vceVVtYXAReHjd5UPvdwLn9VmDJOnIRn2xWJI0YgaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1rtcgSLImyT1JppNcNkeblyXZmWRHkg/2WY8k6XCL+9pxkkXA1cALgT3AtiSTVbVzqM0q4I3AeVX1QJKf7KseSdLs+jwiOAeYrqpdVfUgcD2wYUabVwFXV9UDAFW1r8d6JEmz6DMIlgG7h5b3dOuGPR14epJPJ7ktyZrZdpRkU5KpJFP79+/vqVxJatOoLxYvBlYB5wMbgb9LcurMRlW1uaomqmpi6dKlC1uhJI25PoNgL7BiaHl5t27YHmCyqh6qqq8CX2IQDJKkBdJnEGwDViVZmeQk4CJgckabjzA4GiDJ6QxOFe3qsSZJ0gy9BUFVHQQuBW4G7gZuqKodSa5Isr5rdjNwf5KdwFbgD6rq/r5qkiQdLlU16hqOycTERE1NTY26DEk6oSTZXlUTs20b9cViSdKIGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDVuXkGQ5KXzWSdJOvHM94jgjfNcJ0k6wRzxCWVJ1gLrgGVJ3jm06XHAwT4LkyQtjKM9qvJeYApYD2wfWv9d4HV9FSVJWjhHDIKqugO4I8kHq+ohgCSnASsOPV5SknRim+81gk8keVySxwOfZfAksXf0WJckaYHMNwhOqarvAC8Crq2q5wIX9FeWJGmhzDcIFid5EvAy4KM91iNJWmDzDYIrGDxN7CtVtS3JU4Ev91eWJGmhHO2uIQCq6kbgxqHlXcCL+ypKkrRw5vvN4uVJ/inJvu71oSTL+y5OktS/+Z4aeh8wCTy5e/1Lt06SdIKbbxAsrar3VdXB7nUNsLTHuiRJC2S+QXB/kouTLOpeFwP391mYJGlhzDcIfpvBraPfAO4DXgK8sqeaJEkLaF53DTG4ffQVh6aV6L5h/DYGASFJOoHN94jgWcNzC1XVt4Cz+ilJkrSQ5hsEj+gmmwN+eEQw36MJSdKPsfn+Mf9z4NYkh75U9lLgz/opSZK0kOb7zeJrk0wBL+hWvaiqdvZXliRpocz79E73h98//pI0ZuZ7jUCSNKYMAklqnEEgSY0zCCSpcQaBJDXOIJCkxvUaBEnWJLknyXSSy47Q7sVJKslEn/VIkg7XWxAkWQRcDawFVgMbk6yepd1jgdcAt/dViyRpbn0eEZwDTFfVrqp6ELge2DBLuz8BrgT+p8daJElz6DMIlgG7h5b3dOt+KMnZwIqq+tiRdpRkU5KpJFP79+8//pVKUsNGdrE4ySOAtwNvOFrbqtpcVRNVNbF0qU/IlKTjqc8g2AusGFpe3q075LHAM4B/T/I14Fxg0gvGkrSw+gyCbcCqJCuTnARcBEwe2lhVB6rq9Ko6o6rOAG4D1lfVVI81SZJm6C0IquogcClwM3A3cENV7UhyRZL1fX2uJOnY9PqUsaraAmyZse7yOdqe32ctkqTZ+c1iSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY3rNQiSrElyT5LpJJfNsv31SXYmuTPJLUme0mc9kqTD9RYESRYBVwNrgdXAxiSrZzT7HDBRVc8CbgLe2lc9kqTZ9XlEcA4wXVW7qupB4Hpgw3CDqtpaVd/rFm8DlvdYjyRpFn0GwTJg99Dynm7dXC4B/nW2DUk2JZlKMrV///7jWKIk6cfiYnGSi4EJ4KrZtlfV5qqaqKqJpUuXLmxxkjTmFve4773AiqHl5d26H5HkQuBNwC9V1fd7rEeSNIs+jwi2AauSrExyEnARMDncIMlZwN8C66tqX4+1SJLm0FsQVNVB4FLgZuBu4Iaq2pHkiiTru2ZXAY8Bbkzy+SSTc+xOktSTPk8NUVVbgC0z1l0+9P7CPj9fknR0PxYXiyVJo2MQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWpcr0GQZE2Se5JMJ7lslu0nJ/nHbvvtSc7osx5J0uF6C4Iki4CrgbXAamBjktUzml0CPFBVTwPeAVzZVz2SpNn1eURwDjBdVbuq6kHgemDDjDYbgPd3728CLkiSHmuSJM2wuMd9LwN2Dy3vAZ47V5uqOpjkAPAE4JvDjZJsAjZ1i/+V5J7/Z02nz9x3I1rsd4t9hjb73WKf4dj7/ZS5NvQZBMdNVW0GNj/c/SSZqqqJ41DSCaXFfrfYZ2iz3y32GY5vv/s8NbQXWDG0vLxbN2ubJIuBU4D7e6xJkjRDn0GwDViVZGWSk4CLgMkZbSaBV3TvXwJ8sqqqx5okSTP0dmqoO+d/KXAzsAh4b1XtSHIFMFVVk8B7gA8kmQa+xSAs+vSwTy+doFrsd4t9hjb73WKf4Tj2O/4HXJLa5jeLJalxBoEkNa6ZIDjadBfjIMmKJFuT7EyyI8lruvWPT/KJJF/ufp426lqPtySLknwuyUe75ZXdtCXT3TQmJ426xuMtyalJbkryxSR3J3leI2P9uu7f911JrkvyqHEb7yTvTbIvyV1D62Yd2wy8s+v7nUnOPtbPayII5jndxTg4CLyhqlYD5wKv7vp5GXBLVa0CbumWx81rgLuHlq8E3tFNX/IAg+lMxs1fAh+vqjOBZzPo/1iPdZJlwO8DE1X1DAY3olzE+I33NcCaGevmGtu1wKrutQl417F+WBNBwPymuzjhVdV9VfXZ7v13GfxhWMaPTuXxfuDXR1JgT5IsB34VeHe3HOAFDKYtgfHs8ynALzK4846qerCqvs2Yj3VnMfAT3XePHg3cx5iNd1V9isGdlMPmGtsNwLU1cBtwapInHcvntRIEs013sWxEtSyIbibXs4DbgSdW1X3dpm8ATxxVXT35C+APgR90y08Avl1VB7vlcRzvlcB+4H3dKbF3J1nCmI91Ve0F3gZ8nUEAHAC2M/7jDXOP7cP++9ZKEDQlyWOADwGvrarvDG/rvrA3NvcMJ/k1YF9VbR91LQtsMXA28K6qOgv4b2acBhq3sQbozotvYBCETwaWcPgplLF3vMe2lSCYz3QXYyHJIxmEwD9U1Ye71f956FCx+7lvVPX14DxgfZKvMTjl9wIG585P7U4dwHiO9x5gT1Xd3i3fxCAYxnmsAS4EvlpV+6vqIeDDDP4NjPt4w9xj+7D/vrUSBPOZ7uKE150bfw9wd1W9fWjT8FQerwD+eaFr60tVvbGqllfVGQzG9ZNV9ZvAVgbTlsCY9Rmgqr4B7E7yM92qC4CdjPFYd74OnJvk0d2/90P9Huvx7sw1tpPAy7u7h84FDgydQpqfqmriBawDvgR8BXjTqOvpqY8/z+Bw8U7g891rHYNz5rcAXwb+DXj8qGvtqf/nAx/t3j8V+AwwDdwInDzq+nro788BU914fwQ4rYWxBv4Y+CJwF/AB4ORxG2/gOgbXQB5icPR3yVxjC4TBXZFfAb7A4I6qY/o8p5iQpMa1cmpIkjQHg0CSGmcQSFLjDAJJapxBIEmNMwikY5DktUkePeo6pOPJ20elY9B9g3miqr456lqk48UjAmkOSZYk+ViSO7q579/CYH6brUm2dm1+OcmtST6b5MZunieSfC3JW5N8IclnkjytW//Sbl93JPnU6Hon/R+DQJrbGuDeqnp2Dea+/wvgXuD5VfX8JKcDbwYurKqzGXzL9/VDv3+gqp4J/FX3uwCXA79SVc8G1i9MN6QjMwikuX0BeGGSK5P8QlUdmLH9XAYPOvp0ks8zmP/lKUPbrxv6+bzu/aeBa5K8isFDVaSRW3z0JlKbqupL3WP/1gF/muSWGU0CfKKqNs61i5nvq+p3kjyXwYN0tid5TlXdf7xrl46FRwTSHJI8GfheVf09cBWDaZ6/Czy2a3IbcN7Q+f8lSZ4+tIvfGPp5a9fmp6vq9qq6nMGDZYanD5ZGwiMCaW7PBK5K8gMGs0D+LoNTPB9Pcm93neCVwHVJTu5+580MZrkFOC3JncD3gUNHDVclWcXgaOIW4I6F6Yo0N28flXrgbaY6kXhqSJIa5xGBJDXOIwJJapxBIEmNMwgkqXEGgSQ1ziCQpMb9L/yULSGVRraDAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_layers = 2\n",
    "batch_size = 20\n",
    "steps = 100\n",
    "\n",
    "dispatch_id = ct.dispatch(variation_circuit)(n_layers, steps, batch_size)\n",
    "result = ct.get_result(dispatch_id=dispatch_id, wait=True)\n",
    "loss_history, trained_params, trained_bias, pred_test = result.result\n",
    "\n",
    "print(\"accuracy on test set:\", accuracy_score(pred_test, y_test))\n",
    "\n",
    "plt.plot(loss_history)\n",
    "plt.ylim((0, 1))\n",
    "plt.xlabel(\"steps\")\n",
    "plt.ylabel(\"cost\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ac4ff70",
   "metadata": {},
   "source": [
    "# Estimating scalability\n",
    "\n",
    "We now estimate the number of circuit evaluations needed for kernel and variational based training. The number of circuit evaluations for kernel-based training depends only on the number of samples. This is because it is needed only when constructing the `kernel_matrix` which needs number of samples ^ 2 circuit evaluations. During predicting as the new samples needed to be compared with each of the training sample we need number of training samples * number of predicting samples.\n",
    "\n",
    "\n",
    "The number of circuit evaluations for variational circuit depends on the number of parameters, samples, step_size during training. This is because, in each optimization step, the partial derivative needs to be evaluated for each parameter and approximately two evaluations are needed per calculation. During prediction, one evaluation is needed per sample.\n",
    "\n",
    "\n",
    "In the above variational-based training, we used 24 parameters for training with 75 samples. We use this as a reasonable estimate of how many parameters are needed for a given number of training samples. We compare the scalability with two cases where the number of variables:\n",
    "\n",
    "1. vary linearly with the number of samples: `n_params = M`. In our case this is overfitting.\n",
    "2. saturates at some point, which we model by setting: `n_params = sqrt(M)`. In our case it is underestimating.\n",
    "\n",
    "For gradient-based training , it is reasonable to assume that the number of steps: `n_step` varies linearly with the number of samples. \n",
    "\n",
    "\n",
    "All these information are plotted with the help of utility functions shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "62fdc7b6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7500"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def circuit_evals_kernel(n_data, split):\n",
    "    \"\"\"Compute how many circuit evaluations one needs for kernel-based\n",
    "    training and prediction.\"\"\"\n",
    "\n",
    "    M = int(np.ceil(split * n_data))\n",
    "    Mpred = n_data - M\n",
    "\n",
    "    n_training = M * M\n",
    "    n_prediction = M * Mpred\n",
    "\n",
    "    return n_training + n_prediction\n",
    "\n",
    "\n",
    "circuit_evals_kernel(n_data=len(X), split=len(X_train) / (len(X_train) + len(X_test)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "9c362c69",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "96025"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def circuit_evals_variational(n_data, n_params, n_steps, shift_terms, split, batch_size):\n",
    "    \"\"\"Compute how many circuit evaluations are needed for\n",
    "    variational training and prediction.\"\"\"\n",
    "\n",
    "    M = int(np.ceil(split * n_data))\n",
    "    Mpred = n_data - M\n",
    "\n",
    "    n_training = n_params * n_steps * batch_size * shift_terms\n",
    "    n_prediction = Mpred\n",
    "\n",
    "    return n_training + n_prediction\n",
    "\n",
    "\n",
    "circuit_evals_variational(\n",
    "    n_data=len(X),\n",
    "    n_params=len(trained_params.flatten()),\n",
    "    n_steps=steps,\n",
    "    shift_terms=2,\n",
    "    split=len(X_train) / (len(X_train) + len(X_test)),\n",
    "    batch_size=batch_size,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "15194aa1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2025"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def model_evals_nn(n_data, n_params, n_steps, split, batch_size):\n",
    "    \"\"\"Compute how many model evaluations are needed for neural\n",
    "    network training and prediction.\"\"\"\n",
    "\n",
    "    M = int(np.ceil(split * n_data))\n",
    "    Mpred = n_data - M\n",
    "\n",
    "    n_training = n_steps * batch_size\n",
    "    n_prediction = Mpred\n",
    "\n",
    "    return n_training + n_prediction\n",
    "\n",
    "\n",
    "model_evals_nn(\n",
    "    n_data=len(X),\n",
    "    n_params=len(trained_params.flatten()),\n",
    "    n_steps=steps,\n",
    "    split=len(X_train) / (len(X_train) + len(X_test)),\n",
    "    batch_size=batch_size,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "9a89a42c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABO60lEQVR4nO3dd3wUdfrA8c+zm00nBAgg1aAUKSGUUAVUugqoKGJF9BTv7J6ep57tvPPOcj89251nwQp2EWxIBxEEQhUp0kKXACEhPVu+vz9ms6QRFshmN8nz9rWvzM7Mzjy7LvPsd+Y730eMMSillFKhxhbsAJRSSqmKaIJSSikVkjRBKaWUCkmaoJRSSoUkTVBKKaVCkiYopZRSISnkEpSITBaRdBFZ7+f6V4rIBhH5RUSmBjo+pZRS1UNC7T4oERkE5ADvGWO6nGDddsAnwGBjzBERaWKMSa+OOJVSSgVWyLWgjDGLgIyS80TkbBGZKSIrReQHETnHu+gW4FVjzBHvazU5KaVULRFyCeo4XgfuNMb0BO4H/uOd3x5oLyI/ishPIjIyaBEqpZSqUmHBDuBERCQW6A98KiLFsyO8f8OAdsD5QEtgkYgkGWMyqzlMpZRSVSzkExRWKy/TGNOtgmV7gGXGGCewQ0R+xUpYK6oxPqWUUgEQ8qf4jDFHsZLPOACxJHsXf4nVekJEErBO+W0PQphKKaWqWMglKBH5EFgKdBCRPSLyO+Ba4Hcishb4BbjEu/r3wGER2QDMB/5kjDkcjLiVUkpVrZDrZq6UUkpBCLaglFJKKQixThIJCQkmMTEx2GEopZSqRitXrjxkjGlcdn5IJajExERSU1ODHYZSSqlqJCI7K5qvp/iUUkqFJE1QSimlQpImKKWUUiEppK5BVcTpdLJnzx4KCgqCHYpSIScyMpKWLVvicDiCHYpSVS7kE9SePXuoV68eiYmJlBiLT6k6zxjD4cOH2bNnD23atAl2OEpVuZA/xVdQUECjRo00OSlVhojQqFEjPbugaq2QT1CAJieljkP/bajarEYkKKWUUiGmMAfSNwZ0F5qgQlRiYiKHDh067e0sWLCAJUuWVEFESinl5XbBZzfC5JGQnxmw3WiCChKXy1Ut+9EEpZSqUsbAN3+ELbNg6BMQFR+wXWmCOoG0tDQ6duzILbfcQufOnRk+fDj5+fkAbNu2jZEjR9KzZ08GDhzIpk2bAJg4cSKfffaZbxuxsbGAlSwGDhzImDFj6NSpEwCXXnopPXv2pHPnzrz++usnjCc2Npa//OUvJCcn07dvXw4cOADAwYMHufzyy+nVqxe9evXixx9/JC0tjddee40XXniBbt268cMPP1TpZ6OUqoMWPw+r3oUBf4SUGwO6q5DvZl7W+P8tLTdvVNdmXN8vkfwiNxPfXl5u+RU9WzIupRUZuUX84YOVpZZ9fGu/E+5zy5YtfPjhh7zxxhtceeWVfP7551x33XVMmjSJ1157jXbt2rFs2TJuu+025s2bV+m2Vq1axfr1633dgidPnkzDhg3Jz8+nV69eXH755TRq1Oi4r8/NzaVv37489dRTPPDAA7zxxhs88sgj3H333dx7770MGDCAXbt2MWLECDZu3Mjvf/97YmNjuf/++0/4PpVSqlLrPoG5T0LSOBj8aMB3F7AEJSIdgI9LzDoLeMwY8+9A7TNQ2rRpQ7du3QDo2bMnaWlp5OTksGTJEsaNG+dbr7Cw8ITb6t27d6l7Vl566SWmTZsGwO7du9myZUulCSo8PJxRo0b5Ypk9ezYAc+bMYcOGDb71jh49Sk5Ojv9vUimlKrNjEXx5GyQOhEteBVvgT8AFLEEZYzYD3QBExA7sBaad7nYra/FEhdsrXd4wJtyvFlNZERERvmm73U5+fj4ej4f4+HjWrFlTbv2wsDA8Hg8AHo+HoqIi37KYmBjf9IIFC5gzZw5Lly4lOjqa888//4T3tDgcDl/XYrvd7ruW5fF4+Omnn4iMjDzp96eUUpU6sAE+ug4anQ3jP4CwiBO/pgpU1zWoIcA2Y0yFQ6rXRHFxcbRp04ZPP/0UsO7qX7t2LWD1wFu50jqVOGPGDJxOZ4XbyMrKokGDBkRHR7Np0yZ++umnU45n+PDhvPzyy77nxYmzXr16ZGdnn/J2lVJ13NH9MGUcOKLg2s8C2imirOpKUFcBH1a0QEQmiUiqiKQePHiwmsKpGlOmTOGtt94iOTmZzp07M336dABuueUWFi5cSHJyMkuXLi3Vaipp5MiRuFwuOnbsyIMPPkjfvn1POZaXXnqJ1NRUunbtSqdOnXjttdcAGD16NNOmTdNOEkqpk1eYDVPHQf4RuPYTiG9VrbsXY0xgdyASDuwDOhtjDlS2bkpKiilbsHDjxo107NgxgBEqVbPpvxEVEG4nTB0P2xfANZ9Au6EB25WIrDTGpJSdXx29+C4EVp0oOSmllAoRxsDX98C2uTD6pYAmp8pUxym+qznO6T2llFIhaNFzsPoDGPQn6HlD0MIIaIISkRhgGPBFIPejlFKqiqyZCvOfgq5XwQV/CWooAT3FZ4zJBY5/U49SSqnQsW0+zLgT2gyCMS9DkEfL16GOlFJKwW/r4ePrIaG9916n8GBHpAlKKaXqvKy91r1OEbFw7acQWT/YEQGaoELORRddRGZm5mlvJzU1lbvuugs48YjmX375JU8++SQATzzxBP/6178AeOyxx5gzZ85px1KXlRw4+Oabby41HNXJOHjwICNHjqzK0JSyFGRZyakw20pO9VsGOyKfGjdYbE1ijMEYg+0kxqz69ttvq2Q7KSkppKRYtxUsWLCA2NhY+vfvX+G6zz77LDNmzCg3vzhpBYrL5SIsrOq+glW9var25ptvnvJrGzduTLNmzfjxxx8599xzqzAqVae5iuCTCXBos3Wv0xlJwY6oFG1BncCDDz7Iq6++6nte3MLIyclhyJAh9OjRg6SkJN8oEmlpaXTo0IEJEybQpUsXdu/eXeF2c3JyuPHGG0lKSqJr1658/vnnwLFChRVt55lnniEpKYnk5GQefPBBAM4//3yKb24+dOgQiYmJgJWURo0adcKSG7/++isREREkJCSUi7Hkr//ExEQef/xx3/stLi2Sm5vLTTfdRO/evenevXupz2HgwIH06NGDHj16+FpwFZUcKSk2NpZ7772Xzp07M2TIEIpHF3njjTfo1asXycnJXH755eTl5fli/P3vf0+fPn144IEHWL58Of369aN79+7079+fzZs3A/DOO+9w6aWXMmzYMBITE3nllVd4/vnn6d69O3379iUjI6NcLJ9++ildunQhOTmZQYMGAeB2u7n//vvp0qULXbt29Q0v9eSTT9KrVy+6dOnCpEmTqOgG+JL/r45XNmXbtm307duXpKQkHnnkEV+pFrBKs0yZMqXcdpU6JcbAV3dbN+KOfhHaDgl2ROWE7s/Ninz3IPz2c9Vu84wkuPDp4y4eP34899xzD7fffjsAn3zyCd9//z2RkZFMmzaNuLg4Dh06RN++fRkzZgxgled49913Kx266G9/+xv169fn55+t93PkyJFy65Tcznfffcf06dNZtmwZ0dHRFR5QK5KYmFhpyY0ff/yRHj16+LWthIQEVq1axX/+8x/+9a9/8eabb/LUU08xePBgJk+eTGZmJr1792bo0KE0adKE2bNnExkZyZYtW7j66qt9B+eyJUdKys3NJSUlhRdeeIEnn3ySv/71r7zyyiuMHTuWW265BYBHHnmEt956izvvvBOAPXv2sGTJEux2O0ePHuWHH34gLCyMOXPm8PDDD/uS//r161m9ejUFBQW0bduWZ555htWrV3Pvvffy3nvvcc8995SK5cknn+T777+nRYsWvtOur7/+OmlpaaxZs4awsDDf/4c77riDxx57DIDrr7+er7/+mtGjRx/3s6ysbMrdd9/N1Vdf7RuuqlhKSgqPPPKIX/+vlDqhBf+EtVPhvAeh+3XBjqZC2oI6ge7du5Oens6+fftYu3YtDRo0oFWrVhhjePjhh+natStDhw5l7969vl/BZ5555gnH1ZszZ44v6QE0aNCg3DoltzNnzhxuvPFGoqOjAWjYsGGVvL/9+/fTuHFjv9YdO3YscKzkCMCsWbN4+umn6datm2809l27duF0OrnllltISkpi3Lhxpa69lC05UpLNZmP8+PEAXHfddSxevBiwksvAgQNJSkpiypQp/PLLL77XjBs3DrvdDlgD8I4bN44uXbpw7733llrvggsuoF69ejRu3Jj69ev7EkhSUpLv/ZR07rnnMnHiRN544w3cbjdg/X+49dZbfacSi/8/zJ8/nz59+pCUlMS8efNK7bciZcumFO9/6dKlvhIu11xzTanXNGnShH379lW6XaX8sup9WPgMdLsWzn8w2NEcV81qQVXS0gmkcePG8dlnn/Hbb7/5Dp5Tpkzh4MGDrFy5EofDQWJioq9UxvEGhz1Z/mynZGmPE5XqqEhUVBRZWVl+rVtcdqRkmQ9jDJ9//jkdOnQote4TTzxB06ZNWbt2LR6Pp1QZkJP5fIpLi0ycOJEvv/yS5ORk3nnnHRYsWFDh9h599FEuuOACpk2bRlpaGueff365+MFKhMXPbTab7/2U9Nprr7Fs2TK++eYbevbs6RuhvqyCggJuu+02UlNTadWqFU888cQpl02pTEFBAVFRUSdcT6lKbZ1jndo76wLr1F6Q73WqjLag/DB+/Hg++ugjPvvsM9+v26ysLJo0aYLD4WD+/Pns3HlylUSGDRtW6tpWRaf4yq7/9ttv+669FJ9aKlnao2SZ+ZIqK7nRsWNHtm7delKxlzRixAhefvll3zWX1atXA9bn06xZM2w2G++//76vBXIiHo/H9z6mTp3KgAEDAMjOzqZZs2Y4nc5Kr8NkZWXRokULwLrudDq2bdtGnz59ePLJJ2ncuDG7d+9m2LBh/O9///MllIyMDF8ySkhIICcn57j/H/zRt29f3ynJjz76qNSyX3/9lS5dupzytpViz0r4eAI06QhXvgd2R7AjqpQmKD907tyZ7OxsWrRoQbNmzQC49tprSU1NJSkpiffee49zzjmnwtc+9thjFfaQe+SRRzhy5IjvIvz8+fMrjWHkyJGMGTOGlJQUunXr5usKfv/99/Pf//6X7t27c+jQoQpfW1nJjUGDBrF69eoKL+r749FHH8XpdNK1a1c6d+7Mo49aZaBvu+023n33XZKTk9m0aZPfraaYmBiWL19Oly5dmDdvnu+6zt/+9jf69OnDueeee9zPGuCBBx7goYceonv37n61Sirzpz/9iaSkJLp06UL//v1JTk7m5ptvpnXr1nTt2pXk5GSmTp1KfHw8t9xyC126dGHEiBH06tXrlPf573//m+eff56uXbuydetW6tc/dj/K/Pnzufjii0/rPak6LH0TTLkcYhpZdZ0i44Id0QkFvNzGydByG8Fx9913M3r0aIYODc6IxSXFxsbW6VL1eXl5REVFISJ89NFHfPjhh76ekYMGDWL69OnlrlfqvxF1Qpm74K0R4HHBTTOtyrghJJjlNlSIe/jhh1m2bFmww1DAypUrueOOOzDGEB8fz+TJkwHrRt0//vGPFXamUapSOenw3qVQlAs3fhtyyaky2oJSqobTfyPquAqy4J2L4dBWmPAltD71qt2BpC0opZSqS5z5MPUqSN8IV38cssmpMpqglFKqtnE74ZMbYNdSuPzNoFXEPV2aoJRSqjbxeODL22DL93Dx/0HSFcGO6JRpN3OllKotjIGZD8LPn8DgR6DXzcGO6LRoggoxwS63cbr+8Y9/VMl2agItpaFCzsJnYPn/oO/tMLD82Js1jSaoADLG+IYh8te3335LfHz8aW8nJSWFl156CThxgnr22We57bbb/NruiW5+DVSCOt2bbgPtzTffrHB0dn+ULKWh1Clb9j9rANjka2D430N6CCN/BTRBiUi8iHwmIptEZKOI9Avk/gKhrpXbqKjExDvvvMOYMWMYPHgwQ4YMIT8/n6uuuoqOHTty2WWX0adPH1JTU3nwwQfJz8+nW7duXHvtteXes5bS0FIaKkDWfgzfPQAdLoYxL8NJ1I4LZYHuJPEiMNMYc4WIhAPRp7OxZ5Y/w6aMTVUTmdc5Dc/hz73/fNzlda3cRkUlJsAqkbFu3ToaNmzI888/T3R0NBs3bmTdunW+1z/99NO88sorrFmzpsJYtJSGltJQAbB5Jnz5B0gcCFdMBnvt6fsWsDQrIvWBQcBbAMaYImNMZqD2Fyh1rdxGRSUmwBqstnifixYt4rrrrPoxXbt2pWvXrn7tS0tpaCkNVcXSfoRPb7Dq2l01FRyRJ35NDRLIVNsGOAi8LSLJwErgbmNMbsmVRGQSMAmgdevWlW6wspZOINWlchvHKzFRVe+pJC2lUXo/WkpDnZT9a+HDq6B+K7ju8xox+OvJCuSJyjCgB/BfY0x3IBcoVxnLGPO6MSbFGJPib+G86laXym1UVGKirEGDBjF16lTAav2sW7fOt8zhcOB0Oivcl5bS0FIaqooc3gYfXA4RcdYQRjEJwY4oIAKZoPYAe4wxxaOQfoaVsGqculRuo6ISE2X94Q9/ICcnh44dO/LYY4/Rs2dP37JJkybRtWvXCjtJaCkNLaWhqkDWXmvwV2Os5FS/ZbAjChxjTMAewA9AB+/0E8Bzla3fs2dPU9aGDRvKzVNV66677jKzZ88+5defd955ZsWKFSdcLyYm5pT3URvk5uYaj8djjDHmww8/NGPGjPEtGzhwoMnIyDil7eq/kTok55AxL/cy5qkWxuxdHexoqgyQairICYHu7nEnMMXbg287cGOA96dOgZbbqB5aSkOdlsJsmHIFHEmD67+A5t2CHVHAabkNpWo4/TdSBzgLYOqVkLYYxn8A51wU7IiqlJbbUEqpmshVCJ9MgB0L4dLXal1yqkztuN1YKaVqI1cRfDrRGpl81L+h29XBjqhaaYJSSqlQ5HbCZzfC5m/hon9BSt27hK8JSimlQo3bBZ/fDJu+hpHPQO9bgh1RUGiC8kN+fj7nnXdeqaF/Aq0qRwUvKipi0KBBFd47lJaWVu03iJYcJFUpVYbHDdNuhQ1fwvCnoO/vgx1R0GiC8sPkyZMZO3asb0y46lCVCSo8PJwhQ4bw8ccfV9k2IfRLYChV43jcVjXc9Z/B0Ceg/x3BjiioNEH5YcqUKVxyySWAdWPzHXfcQYcOHRg6dCgXXXSRb/ib4lIZYBUMLB4rrrIyEWPHjmXkyJG0a9eOBx54AKBc2YqyrZx//etfPPHEE4BV5uHee+8lJSWFjh07smLFCsaOHUu7du1KjY7tTzmH7du30717d1asWMG2bdsYOXIkPXv2ZODAgWzaZI0iX7YExsSJE7nrrrvo378/Z511VqmhgJ577jl69epF165defzxx0/141eqbvB4YMadsO4jqxrugHuDHVHQnVQ3cxGxAbHGmKMBiqdSv/3jHxRurNpyGxEdz+GMhx8+7vKioiK2b9/uq7M0bdo0Nm/ezIYNGzhw4ACdOnXipptuqnQf55xzznHLRKxZs4bVq1cTERFBhw4duPPOO8uVrahoNO6SwsPDSU1N5cUXX+SSSy5h5cqVNGzYkLPPPpt7772XRo0a0aVLF1asWHHcbWzevJmrrrqKd955h+TkZIYMGcJrr71Gu3btWLZsGbfddhvz5s0DSpfAmDhxIvv372fx4sVs2rSJMWPGcMUVVzBr1iy2bNnC8uXLMcYwZswYFi1a5Ku9pJQqweOBr++GNVPg/Idg0J+CHVFIOGGCEpGpwO8BN7ACiBORF40xzwU6uFBw6NChUhVuFy1axNVXX43dbqd58+YMHjz4hNvIysrihhtuYMuWLYhIqcFUhwwZ4huTrVOnTuzcuZNWrVqdVIzFdaiSkpLo3Lmzb7zAs846i927d9OoUSPsdjvh4eFkZ2dTr169Uq8/ePAgl1xyCV988QWdOnUiJyeHJUuW+AbGBSgsLPRNlyyBAVbrzGaz0alTJ1/JkVmzZjFr1iy6d+8OWAUat2zZoglKqbKMgW/vg1XvWWXazwtO1YZQ5E8LqpMx5qiIXAt8hzUi+Uqg2hNUZS2dQImKivK7jMXxSl/4WybieKUZSm637LZLbqNkmYni5yW3V1hYSGRk+Xox9evXp3Xr1ixevJhOnTrh8XiIj48/buHBsqU3Su6zeGQSYwwPPfQQt956a4XbUEphJafvHoDUyXDuPdapvVpQqr2q+HMNyiEiDuBSYIYxxgmEzvhIAdagQQPcbrcvKQwaNIiPP/4Yt9vN/v37S41CXrL0RfEpPDi1MhEly1Y0bdqU9PR0Dh8+TGFhIV9//fVJv4/Dhw+TkJCAw+Eotyw8PJxp06bx3nvvMXXqVOLi4mjTpg2ffvopYCWbtWvXntT+RowYweTJk8nJyQFg7969pKenn3TcStVaxsD3D8Py16HfHVanCE1OpfiToP4HpAExwCIRORMIyjWoYBk+fLiv+utll11Gu3bt6NSpExMmTKBfv36+9R5//HHuvvtuUlJSSp0CO5UyESXLVjgcDh577DF69+7NsGHDKi1JcTwnKucQExPD119/zQsvvMCMGTOYMmUKb731FsnJyXTu3Jnp06ef1P6GDx/ONddcQ79+/UhKSuKKK644bk0qpeocY2D2o/DTf6DP72H43zU5VeCUBosVkTBjTJX3MQ7VwWJXrVrFCy+8wPvvv19u2cSJExk1ahRXXHFFECLz39ixY3n66adp3759sENRVSwU/o2ok2AMzH0SFj8PvW62Romo48nplAeLFZEI4HIgscz6T1ZZdCGuR48eXHDBBbjd7mq9F6qqFBUVcemll2pyUioULPinlZx6ToQLn6vzyaky/nSSmA5kYXWMKDzBurXW8bqSn27p8eoQHh7OhAkTgh2GUmrhs7DwGeh+HVz8Atj0VtTK+JOgWhpjRgY8kkoYYxD9laFUOaFUz02dwA//B/OfguSrYfRLmpz84M8ntEREkgIeyXFERkZy+PBh/YeoVBnGGA4fPlzhrQMqxPz4onXdKWkcXPIq2GrepYJg8KcFNQCYKCI7sE7xCWCMMV0DGplXy5Yt2bNnDwcPHqyO3SlVo0RGRtKyZctgh6Eqs/RVmP0YdL7MKjioyclv/iSoCwMeRSUcDgdt2rQJZghKKXVqfvqvda9TxzEw9g2waxHzk3HCT8sYs1NEkoGB3lk/GGP8umtTRNKAbKxhklwVdSNUSqlaadG/YN7f4JxRcMVksJe/SV5V7oTXoETkbmAK0MT7+EBE7jyJfVxgjOmmyUkpVScU3+c072+QdCWMe1eT0ynyp735O6CPMSYXQESeAZYCLwcyMKWUqnGMgZkPwbL/Qo8bYNQLes3pNPjTi0+wTtEVc3vn+cMAs0RkpYhMqnDjIpNEJFVEUrUjhFKqxvK44au7rOTU5w8w+kVNTqfJnxbU28AyEZnmfX4p8Jaf2x9gjNkrIk2A2SKyyRizqOQKxpjXgdfBGurIz+0qpVTocLvgy9/Dz5/CwPtg8KM6QkQV8KeTxPMisgCruznAjcaY1f5s3Biz1/s33ZvgegOLKn+VUkrVIK5C+Owm2PS1lZgG3R/siGqN4yYoEYnz1oFqiDWaeVqJZQ2NMRmVbVhEYgCbMSbbOz2cOjR+n1KqDijKg0+uh61zYOTT0PcPwY6oVqmsBTUVGIU1Bl/JU2/ifX7WCbbdFJjmHaIoDJhqjJl56qEqpVQIKcyGD6+GtMXW0EU9bwh2RLXOcROUMWaU9+8p3SVrjNkOJJ9iXEopFbryj8CUcbB3lXUDbtdxwY6oVvLnPqi5/sxTSqk6IfcQvDsa9q+FK9/T5BRAlV2DigSigQQRacCxruVxQItqiE0ppULL0f3w3iWQuROu/hDaDg12RLVaZdegbgXuAZpjXYcqTlBHgVcCG5ZSSoWYIzvhvTFWC+q6zyFxwIlfo05LZdegXgReFJE7jTE6aoRSqu46vA3eHQNF2XD9l9CqV7AjqhP8uQ/qZRHpAnQCIkvMfy+QgSmlVEg4sME6rWfccMPX0KxaKg0p/EhQIvI4cD5WgvoWq/zGYkATlFKqdtu3Gt4fC/ZwmPg1NO4Q7IjqFH/G4rsCGAL8Zoy5EavreP2ARqWUUsG26yfrtF54LNz0nSanIPAnQeUbYzyAS0TigHSgVWDDUkqpINq+EN6/DGIaW8mp4YnGJVCB4M9gsakiEg+8gdWbLwer3IZSStU+v0yDLyZBo7ZWh4h6TYMdUZ3lTyeJ27yTr4nITCDOGLMusGEppVQQ/PQazHwQWvWx7nOKbhjsiOo0fzpJDKpoXtmyGUopVWN5PDD3CfjxRatE++VvgiMq2FHVef6c4vtTielIrJIZK4HBAYlIKaWqk6sIZtwB6z6GlN/BRc9pocEQ4c8pvtEln4tIK+DfgQpIKaWqTWE2fHw9bJ8Pgx+BgfdrocEQ4k8Lqqw9QMeqDkQppapV9gGYOg5+Ww+XvArdrwt2RKoMf65BvcyxelA2oBuwKoAxKaVUYB3aCh+MhdyDcPVH0H54sCNSFfCrm3mJaRfwoTHmxwDFo5RSgbUnFaZeaU3f8DW07BnceNRx+XMN6t3qCEQppQLu1+/h04kQ2wSu+wIanR3siFQlKqsH9TOlS737FgHGGKMjJiqlao5V78FX98AZXeDaz6wkpUJaZS2oUdUWhVJKBYoxsOg5mP8UnD3YqoIbUS/YUSk/VFYPamdV7EBE7FjXsfYaYzTpKaWqj8cN39wHK9+GrlfBmJchLDzYUSk/nXCwWBHpKyIrRCRHRIpExC0iR09iH3cDG089RKWUOgXOfOsep5Vvw4B74bLXNDnVMP6MZv4KcDWwBYgCbgZe9WfjItISuBh481QDVEqpk5aXYRUZ3PwtXPgcDH1Cb8CtgfxJUBhjtgJ2Y4zbGPM2MNLP7f8beADwnFp4Sil1kjJ3weQRsG8NjHsH+kwKdkTqFPlzH1SeiIQDa0TkWWA//p0aHAWkG2NWisj5law3CZgE0Lp1a39iVkqpiv32M3xwhXV67/ppkHhusCNSp8GfFtT13vXuAHKxihVe7sfrzgXGiEga8BEwWEQ+KLuSMeZ1Y0yKMSalcePGfgeulFKlbJ0Db18EYoObZmpyqgX8SVA9se57OmqM+asx5o/eU36VMsY8ZIxpaYxJBK4C5hljdLArpVTVMgZ++i9MGQfxreHm2dC0U7CjUlXAnwQ1GvhVRN4XkVEicioDzCqlVNVzO+Hre6wigx0ugpu+h/otgx2VqiInTFDGmBuBtsCnWL35tonISfXKM8Ys0HuglFJVKi8D3r8MVr4DA/4IV74PEbHBjkpVIb9aQ8YYp4h8hzX0URRwKVZ3c6WUqn4HN8PU8XB0H1z2OiSPD3ZEKgD86Y13oYi8g3Uf1OVY9zSdEeC4lFKqYlvmwJtDoSgHJn6tyakW86cFNQH4GLjVGFMY4HiUUqpixsCy1+D7h6FJZ7j6Q4hvFeyoVAD5cw3qamA1MBBARKJEREdaVEpVH1cRfHV3ic4QMzU51QH+nOK7BfgM+J93VkvgywDGpJRSxxR3hlj1Lgy8TztD1CH+nOK7HegNLAMwxmwRES2kopQKvPRN8OF4OLofxr4BXa8MdkSqGvmToAqNMUXiHWjRex9URYUMlVKq6myZDZ/dBGGRMPEbaNUr2BGpaubPjboLReRhIEpEhmHdD/VVYMNSStVZxsDS/8DUK6HBmTBpvianOsqfBPUgcBD4GbgV+BZ4JJBBKaXqKFcRfHUXfP+QjgyhTnyKzxjjAd7wPpRSKjByD8MnE2DnYhh4P1zwF7D5VRFI1VI6rp5SKvhKdYZ4E7qOC3ZEKgRoglJKBdfmmfDFLeCIghu/hZYpwY5IhYjjtp9F5H3v37urLxylVJ3hccPcJ62WU8M2cMs8TU6qlMpaUD1FpDlwk4i8B0jJhcaYjIBGppSqvXIOwuc3wY5F0OMGuPBZcEQGOyoVYipLUK8Bc4GzgJWUTlDGO18ppU7Orp/g04mQfwQu+Q90vzbYEakQddwEZYx5CXhJRP5rjPlDNcaklKqNiivfzn4U6reCm+fAGUnBjkqFMH+6mf9BRJLxDhYLLDLGrAtsWEqpWqUwG6bfARu+hHNGwSWvQlR8sKNSIc6fwWLvAqYATbyPKSJyZ6ADU0rVEukb4fULYONXMOxJGP+BJiflF3+6md8M9DHG5AKIyDPAUuDlQAamlKoF1n1ilckIj4UbZkDigGBHpGoQfxKUAO4Sz92U6dGnlFKluAph5kOQ+ha07g/j3oZ6WohbnRx/EtTbwDIRmeZ9finw1oleJCKRwCIgwrufz4wxj59inEqpmiJzF3xyA+xbBf3vhCGPg90R7KhUDeRPJ4nnRWQBUNw2v9EYs9qPbRcCg40xOSLiABaLyHfGmJ9OPVylVEjbMge+uNm6CXf8B9BxdLAjUjWYX0MdGWNWAatOZsPGGAPkeJ86vA+tI6VUbeRxw8JnYeEz0LQzXPkeNDo72FGpGi6gY/GJiB3rJt+2wKvGmGUVrDMJmATQunXrQIajlAqE3MNWq2nbPEi+Bi7+PwiPDnZUqhYI6Fj2xhi3MaYb0BLoLSJdKljndWNMijEmpXHjxoEMRylV1fashP8NgrQfYfSLcOl/NDmpKlNpghIRu4jMP92dGGMygfnAyNPdllIqBBgDy16HySOsmk2/+x56TgTRDr6q6lSaoIwxbsAjIvVPdsMi0lhE4r3TUcAwYNOpBKmUCiE5B2HqePjuT3D2YJi0EJp3D3ZUqhby5xpUDvCziMwGcotnGmPuOsHrmgHveq9D2YBPjDFfn3KkSqng+3UWTL8NCo5aI5D3nqStJhUw/iSoL7yPk+Idr09/VilVGzjzYdajsOINaNoFJsyApp2CHZWq5fy5D+pd7ym61saYzdUQk1IqlOxfZ1W8PbgJ+t4OQx7T2k2qWvgzWOxoYA0w0/u8m4jMCHBcSqlg83hgycvw5hDIz4Trp8HIf2hyUrhzcsmYMoV9jzwS0P34c4rvCaA3sADAGLNGRLRYoVK12dF9MO33sGOhVR5j9EsQ0yjYUakgK9q5k4wpU8j6YhqenBwik5Lw5OVhiw7MrQX+JCinMSZLSl8I9QQkGqVU8G2YDjPuAneRlZh6TNCOEHWY8XjI/XEJGR+8T+7CReBwEDdiBA2vv46o5OSA7tufBPWLiFwD2EWkHXAXsCSgUSmlql9hDsz8M6z+wOo2PvZNSGgb7KhUkLhzcsn68kuOfPABRWlp2BMSSLj9duLHX4mjSZNqicGfBHUn8BeswV8/BL4H/hbIoJRS1WxPKnx+MxxJg4H3w/kP6gjkdVRRWhoZU6aS9cUXeHJziUzuSvPnniNuxHAkPLxaY/GnF18e8BdvoUJjjMkOfFhKqWrhdsHi52HB0xDXHG78Fs7sH+yoVDUzHg+5ixeT8cEH5C76wTqNd+FIGl53HVFduwYtrhMmKBHpBUwG6nmfZwE3GWNWBjg2pVQgHUmDL26F3T9BlyusQV61FHud4s7JIeuLaRyZMoWinTuxN04g4c47aHDllYSFwNio/pziewu4zRjzA4CIDMAqYhi8tKqUOnXGWKXYv7nP6vww9g3oemWwo1LVqHD7Do5MmULWtGl48vKISk6m+Z13Ejd8WLWfxquMPwnKXZycAIwxi0XEFcCYlFKBkpcB3/4J1n8GrfrC2NehwZnBjkpVA+NykT1/PpkffUzujz8iDgdxF11Ig+uuIyopKdjhVei4CUpEengnF4rI/7A6SBhgPN57opRSNYQxsOFLKznlH4ELHoEB94I9oCXhVAhw7t9P5qefkfnZZ7jS0wk74wwS7rrTOo2XkBDs8CpV2bfz/8o8f7zEtFbGVaqmOLrfOp23+Rto1s0aEeKM0PzFrKqGcbvJ+eEHMj/+hJyFC8EYYgYN5IwnHid20CAkrGb8MDlulMaYC6ozEKVUFTMGVr1nDfLqLoRhT1pj6WmrqdZypqeT9cUXHPnkE1z79mNPSKDRLbcQP24c4S1bBDu8k+ZPL754YAKQWHJ9P8ptKKWCJWM7fHU37FgEZw6AMS9Bo7ODHZUKAOPxkLt0KZkff0L2vHngchHTvx9NH/gz9YYMRhw19342f35KfQv8BPyMDnGkVGjzuOGn/8C8p6wbbUf9G3rcYFW9VbWKKyPD21r6FOeuXdjj42l4wwQajBtHeGJisMOrEv4kqEhjzB8DHolS6vQc+AWm3wH7VkH7kXDx81C/5p3WUcdnjCFvxQqrtTRrFsbpJDolhcZ33km9EcOxhVAX8argT4J6X0RuAb7GGu4IAGNMRsCiUkr5z1UIP/yf9YiMh8vfgi6X6wCvtYjryBGOzpjBkY8/oWj7dmxxccRfdRUNxl9JRNvaO16iPwmqCHgOazy+4t57BtCSG0oF2+7lVqvp0GboOh5G/FPLYtQSxu0m98cfyfz8C3LmzcM4nUQlJ9PsH/8g7sKR2KKigh1iwPmToO4D2hpjDgU6GKWUnwpzYN7fYdlrENcCrvkU2g8PdlSqChSlpZH5xTSypk/HdeAA9gYNaHDN1dQfO5bIDh2CHV618idBbQXyTnbDItIKeA9oitXiet0Y8+LJbkcpVca2eVYPvcxd0OsWGPo4RNQLdlTqNHhyczn6/Swyv/ic/NSVYLMRO3Ag9f/yMPXOPz+khh+qTv4kqFxgjYjMp/Q1qBN1M3cB9xljVolIPWCliMw2xmw49XCVqsPyMmDWI7BmCjRqCzd+pyOP12DGGPJXrybz8885+t1MTF4e4YmJNL7vj9QfcwmOptVTcymU+ZOgvvQ+TooxZj+w3zudLSIbgRaAJiilToYxsP5zmPkQ5B2GAX+E8/4MjshgR6ZOgfNAOlnTp5P1xRcUpaVhi44m7qILiR97OVHduyHaucXHn3pQ757uTkQkEegOLKtg2SRgEkDr1q1Pd1dK1S7718F3f4ZdS6BZMlz3mfVX1SimqIjs+QvI/OJzcn9YDB4P0SkpNLr1VuKGD8MWExPsEEOSPyNJ7KCCsfeMMX714hORWOBz4B5jzNEKtvM68DpASkqKjvGnFFin8+b9HVa+DVENYPSL0P16sNmDHZnykzGGwo0brdbSjK9wHzlCWNOm1tBDl11aa26mDSR/TvGllJiOBMYBDf3ZuIg4sJLTFGPMFycfnlJ1jNtlJaV5f4fCbOg9ySq/HtUg2JEpPzn37iXr62/I+moGRVu3IQ4HsUOGEH/5WGL690fs+iPDX/6c4jtcZta/RWQl8FhlrxPrROpbwEZjzPOnHqJSdcSOH6zTeem/QJtBMPIZaNop2FEpP7izsjg683uyvpph9cIDonr25IwnnqDeiOGENdAfGKfCn1N8PUo8tWG1qPxpeZ0LXA/8LCJrvPMeNsZ8e7JBKlWrZe6G2Y/CL9Ogfiu48j3oOEZHgghxnsJCchYs5OjXX5GzYCHG6ST8rLNofM/dxI0aRXjLlsEOscbzJ9GUrAvlAtKAE9aHNsYsBvRfmFLH48yHJS/DD88DBs5/CPrfBeHRwY5MHYfxeMhLTeXoV19xdOb3eLKzsTdOoME11xA3ZjSRnTppL7wq5M8pPq0LpVRVMgY2fQ3fP2zdbNvpEhj+d4jXXqyhquDXXzn61Vdkff0Nrv37keho4oYNI270aGL69qkxBQBrGn9O8UUAl1O+HtSTgQtLqVoqfRPM/DNsXwCNO8KEGXDWecGOSlXA+dtvHP3mG7JmfEXh5s1gtxMz4Fya3Hcf9QZfgC1aW7qB5k/anw5kASspMZKEUuok5GfCwmdg2f8gIhYufA5SbtLqtiHGdfgw2bNnc/S7meQtXw7GEJnclaaPPELchSMJa6QD8VYnf/51tDTGjAx4JErVRh4PrPkA5vzVGgWi50QY/AjEJAQ7MuXlysgge9Zsjn4/k7xly8HjIbxNGxJuu436o0fp/UpB5E+CWiIiScaYnwMejVK1hTGwdS7M/Sv8tg5a9YXrPofm3YIdmcKqr5Q9ezbZM2eSu2w5uN2En3kmjW6dRNzIC4lo3047O4QAfxLUAGCid0SJQqyeecYY0zWgkSlVU+1cCnOftIYnim8NY9+ApHHabTzIXEeOkD1nDtkzvyf3p5/A7cZxZmsa3XwzcReOJKJDB01KIcafBHVhwKNQqjbYv9YaAWLLLIhtChf9C3rcAGF1s1RCKHBnZpI9dy5Hv5tpJSWXC0fr1jT63e+spHTOOZqUQpg/3cx3VkcgStVYh7bA/KesG20j42HoE9D7Vr2fKUjcWVlkz5nL0e9nkrtkqZWUWrWi0Y03Um/kCL1XqQbRLkRKnarM3VbPvDVTISwSBv0J+t0BUfHBjqzOcR06RPb8+WTPmWMlJacTR4sWNJp4A/VGXkhkZ01KNZEmKKVOVs5B+OH/IPUt63mfW60aTbGNgxtXHVO0cyfZc+aSPXcu+atXgzE4WrSg4YTriRs5ksguXTQp1XCaoJTyV34mLH0Flv4HXAXQ7RqrcGB8q2BHVicYYyhY/wvZc+eQM3cuhVu2AhDRqSMJd9xOvaFDiWjfXpNSLaIJSqkTKcqD5f+Dxf+GgkzoPBYueBgS2gU7slrPFBWRu2IFOXPnkj13Hq4DB8BuJzolhaYPX0m9IYNxtGgR7DBVgGiCUup4XEWw6l1Y9BzkHIB2I6ybbJvpHRaB5M7JJXfxD2TPmUvOwoV4srORyEhiBw4gdsg9xJ53npavqCM0QSlVltsJP38KC/5pDeZ65rlWCYzWfYMdWa3lOnjQ6uQwdy55S5ZinE7sDRpQb9gw6g0dQky/ftiiooIdpqpmmqCUKlaUC6veh6WvQtYuaJYMo16As4foTbZVzHg8FPzyCzmLFpGzaBEF6362Ojm0bEmDa6+l3tAhRHXvrtVn6zhNUErlHoLlr1uP/CPQuh9c9Cy0H6mJqQq5s7LI/fFHchYuImfxYtyHD4MIUV27knDnHdQbMlSHGFKlaIJSdVfGDqu1tPoDcOVDh4vh3LuhdZ9gR1YrGGMo3LzZSkiLFpG/Zg243djr1ydm4EBizxtEzLnnEtawYbBDVSFKE5Sqe/avhR9ftEZ+EDskj4f+d0Pj9sGOrMZz5+SSu3QJuYsWkbPoB6vXHRDZqRONJt1C7KBBRHXtqqfulF80Qam6wRirSOCPL8L2+RARB/3vhD5/gLhmwY6uxjLGULR9u6+VlLdyJTid2GJjiTn3XGIHDSJm4AAcTZoEO1RVAwUsQYnIZGAUkG6M6RKo/ShVKbcLNk63EtP+tRB7Bgz9K6TcCJH1gx1djeTOyiJ32TJyly4ld9EPOPfuBSCiXTsa3TCBmEGDiO7eHXE4ghypqukC2YJ6B3gFeC+A+1CqYkV5sGaKNfLDkTRo1A7GvAxdx0NYRLCjq1E8hYXkr15jJaSlSylYvx48HiQ6mpg+fWh0yy3EDhqIo3nzYIeqapmAJShjzCIRSQzU9pWqUF4GLH/DGvkh7zC07AXDn4IOF4HNFuzoagTj8VC4aZOVkJYsJW/lSkxBAdjtRCUnk/CHPxDTv591LUlbSSqAgn4NSkQmAZMAWrduHeRoVI21bw2sfAfWfQzOPKuL+Ll3W13GtdvyCRXt2UPukiXkLl1K3tKfcGdmAhDRri3xV44jpl8/onv1wh4bG9xAVZ0S9ARljHkdeB0gJSXFBDkcVZMU5sD6z6zEtG81hEVBl8uh3+3QtFOwowtpriNHyFu23JeUnLt3AxDWpAmx551HzLn9ie7TF0dT7dyggifoCUqpk7Z/rbe19CkUZUPjjnDhs9b1Ja3FVCHXkSPkr1pF3opU8pYvp2DjRjAGW0wM0X360HDCBGL69yP8rLP0RlkVMjRBqZqhMAfWf+5tLa2yCgR2Hgs9J0Kr3noarwxnejr5qankpaaStyKVwi1bAJDwcOs60p13ENOvH1FJSUiYHgZUaApkN/MPgfOBBBHZAzxujHkrUPtTtdT+dd7W0ife1tI5MPIZ6+baKB3RuljRnr3kpa4gLzWV/BWpFO3cCYAtOpqoHj2Iu/hionulEJmUhC08PMjRKuWfQPbiuzpQ21a1XFEurP8CVr4Ne1d6W0uXeVtLfep8a8kYQ9GONF9CyktNxbVvPwC2+vWJ7tmT+PHjrYTUsaO2kFSNpd9cFTp+W28lpXWfQOFRSOgAI5+2ri1F193x2ozTScGvv5K/arUvIbkPHwbAnpBAdK8Uon/3O6JTehHRri2i3elVLaEJSgVXXgZsnGGVudibCvaIY62l1n3rZGvJmZ5O/tq1FKxdS96aNRSs/8W6DwlwNG9O7IBziUpJITolhfDERO3UoGotTVCq+uVnwqZvrMFat88Hj8tqLY34JyRfVadaS56iIgo3bCB/7VryvQmp+HQdDgeRnToSf+U4ort1Iyo5WcubqzpFE5SqHoXZsPk769rStrngLoL41tY9S53HWsUBa3lLwBiDa98+XzLKX7OWgg0bME4nAGHNmxGVnEzUhAlEd+tGRMeO2CJ0WCZVd2mCUoFTlAu/zrRaSltmg6sA4lpA70lWUmrRo1YnJXd2NgUbN1Kwbp0vIbkOHgRAIiOJ7NKZBhOut5JScje9KVapMjRBqarlzLeS0S9fwK/fW8MOxTaFHjdAl7HQsnetHBPPnZlJwYYNvkf+L7/g3LnLt9zRujXR/fr6klFkh/Y6jp1SJ6AJSp0+VyFsnWu1lDZ/C0U5EJ0AyVdbHR7O7A+22lOgznX4sJWIftlAwS+/ULBhg6/kBFgdGSI7dyb+ssuI7NSJyC5dtGqsUqdAE5Q6Nc582PGDlZQ2fQOFWdaNs13GWqfvEgeCvWZ/vYwxuNIP+pKQlZR+8VWJBXCc2ZrIrknEXzWeqM6diejYkbAGegOxUlWhZh9BVPUxBg5uslpK2+bCziXWNaWI+nDOxVZiOut8sNfM01aewkKKtm+n8NdfKdyyhYJff6Vgw0bchw5ZK4gQ3qYN0b17W62iTp2I7NQRe716wQ1cqVpME5Q6vrwMq0z6trmwbT4c9Z7GSugAKTfB2UOgzcAaVQDQuN0U7dpF4ZYtFP66xfq7ZYs1NJDbba3kcBBx1lnEDhjgPUXXmcgOHbDFxAQ3eKXqGE1Q6hi3yxpaaNtcq6W0bxUYj1Ua/azz4ew/w9mDIb5VsCM9Iev0XLrVIvp1i69lVLhtG6aw0FpJBEfrVkS0a0e9EcOJbN+eiHbtCD/zTO3AoFQI0ARV12XuPpaQdiyEgiwQG7ToCYMegLZDoHmPkL2eZNxunPv3U7QjjaIdOyjcsd3XMvIcPepbL6xxYyLat6fB1VcT4U1EEWefhS06OojRK6UqE5pHHRU4hdmw66dj15IO/WrNr9ccOo6GtkOhzXkhN5qD68gRKwmlWYmoKC2NorQdFO3chSkq8q1nq1ePiPbtibvoQiLatSOyfXvC27bVjgtK1UCaoGozjxvSN1pj3O1ZAXtWWh0dMNYI4Wf2t+5PajvEKmMR5JtmPYWFFO3cWT4R7diBOyvr2IoOB+GtWhGemEjMoEGEJyYS0aYN4YmJ2Bs10rHplKolNEHVJkf3e5OR97FvNThzrWVRDaBFCnS+FFr2spKTI6pawzPG4D50iKI9e3Du3Ydzzx6ce/fi3LuHop27cO7bZ/UW9Apr0oTwNm2oN3Ik4W0SfYnI0aKFlpBQqg7Qf+U1VVEe7F9jJaLipFTcy87mgDOSoPu1VlJqmQINzwp4C8kYgzszE+ceK+k49+71JqO93nl7j3VQ8LI3aoSjRQuiunWj/mWX+RJR+JmJ2GO115xSdZkmqJrAmQ+Ht8H+tcdO1x3YAMbbLTr+TKs0RYsUq3V0RhI4Iqs8DE9hIa6DB3EdOIArPR3nbwdw7ivREtqzB09eXqnX2OvXx9GiBRFt2xJ73nk4WrbA0aIF4S1b4mjeXDspKKWOSxNUqDAGcg5YnRYObbEeh7dYzzN3A95TXxFx1iCrA+61WkYtUiC28ent2uPBnZGB05t4XAfScaUfwOmbTsd14ADuzMxyr7XFxOBo2RJHy5ZE9+1DeIsW1vMWViLSG1mVOjnGGNzGjcvj8v0tfpR6bo4zv8w8p8dZarnbuHG6nTg91qPIXVTqr9PjrHB5yWmXx0WRu4jIsEimXzo9YJ+FJqjq5iyAjO1W4jm85VgyOrQFirKPreeIhkZtrcFVu11rTTftAgnt/Rps1Xg8uLOycGdk4M7IwHXkCO6MI7iPZOA6nHGsFZSebo2w7XKV3oAIYQkJhDVpYp2C69EdR5MmhDVpSliTJoQ1bYKjaVNscXHaKUFVO2MMHuMpdSD3GI/v4Fl8YC55wK5omdN4/7qdvgN+8bTb4y6VCNwet3Vw9x7wSy4vnvYlguPMK95OqefF76HE/OoUZgsj3BaOw+7AYXOUmnbYHL7p6LBo6kfUL7VOdFhgz4AENEGJyEjgRcAOvGmMeTqQ+wsJznyrJZRz0Pv3NzhcIiEd2YmvNQQQ1xIS2kK3q6FRO0jwPuo19yUiYwymsNBKOJs348rIwH0k05t4MryJ50iJRJRh9XrzeCoM0RYbS1jTpoQ1aUxM797e6WNJJ6xJE8ISErQjQogqPjh7jAeXcfkO1MUHULfHfWye9+HxlHhe5kBb2UGz7K9x3+vKHHyLE0TxtNvjPhabd7pkfCVjKbnM5XGVSzwVvb46D+I2sWEXO2G2MMIkDLvNmvbN807bbXbCJKzUsoiwCN+8sq8p+dwudhw2R/ltF2/PVvp5uW2VnF/BumW36bBbSSbMFhbSPzADdgQSETvwKjAM2AOsEJEZxpgNgdpnwLiKINebcIr/lkpC6ceWFR4t9VJjwCPReGIS8UR3xN1yMJ7wJnjs8bglFk9+Ee6MbDy7cvFkb8edsw5PdjbunGw8Obne6RzwFrUrx2bDHh+PvUEDwho0IOLss7GnpGBv2ICwBg2xN2hgTTdsiL1hQ+zx8SFdBK/49EbxQcr31+OpeH4Fy0sdnL0H6+KDZfFBtOxBveRrPB5PqYNr2QN72emyB9WSB+WKDtDHOwhXeFAus6z4daEgzBaGw+YodQAvPlDbxV5q2resxLxwezhhYdYB1SY238G/om2VTQKV7aP4gOywOcpNl/1bfLAueUB32B3WfG/CsEntKw9TUwTyJ3JvYKsxZjuAiHwEXAIELEHtWTOL3Xt2Ic4ixF2IOAvBWUSEuIjEjXEWkp2bB24n4iyy/rqcOHBj97gwLifOwiJwuxBXEZKfCwX52IoKMW4Bl2A8grgAt2A8YWDCEE8Y4rFj3I3B1RhcIC4PNqcbcboQA5DpfawtF7exCZ6oCFzRkTijwnFFReCKCsfZIhZXVENscVGYmHDyo8LJigynsF4EBbERFMZGUBAdTmx0GDYb5Be5yCkswmDwmFyMycbDDmJtdsgy5B9ykV/kxIPBY9xWywwPUeE2wJDvdFHgcmGMB4PHe0A3RIULHuOh0OWiyO3xvdZj3BgMjjDwGA9FLjdOz7FEULwNm81YB32PG5c3GRS/3oMHUyKpmJKtyxBllzDfQdkudhx268DoMeKbZ/M+HHY7UWHh2MSG0yXYxIZN7NglDLtEEhPuINrhQMROfqHxHhCPHXxjI8KJCQ/HGBtHC1ze+WHYsGGz2YmPjKBeZAQuN2Tmub3bt/liaBwTRWxkOE43HM5xYsM64Nq9B+Rm9aOJi4ykoAgOZTuxiZUAbN4DfWKjesRFRpJT4OHAUWep9wfQtkkskQ47h3MK2Z9VUO6zatc0logwO+nZBaQfLSy3vMMZ9XDYbRw4WsDB7PLLOzWLw2YT9mXmczinqNzypJb1AdidkUdmXukfcTYbdG5uLd95OJesfCd4Typ4AI/NRtvmcQBsP5jD4cLiVpkHKCIizE2HM6xrqFvTc8grKt1qiw6307aJtXzzb9kUOEv/cIiNDOPsxrEAbNx/lCJX6TMacVEO2iRYPVXX783C5Sn93W8Q7eDMRtbytbszy/3LaBQTTquG0RhjWLsni7Ka1IugeXwULreHn/eWX96sfhRn1I+k0OVmw76j5Za3aBBFk3qR5Be52fhb+eWtG0aTEBvYH7uBTFAtgN0lnu8B+gRwf2y8715a7q34tFbx3NgTbKN81aIwIMz6yjqgKKzEwwFOu6HI4aQo3FVq/rH1hIIIIS+CY49wIS/y2PNCB1hZL8f/N5tTenWb2MAIbg+AANZzEOIiwwmz2Slwesgt9HiXi3e5jVYNYnDY7RzJdZGR6yyx3AYIyS0bYLfZ2HOkgANZRaW2Lwjnd2iKXexs3J/NniMF3mXWax02O5d0a4ndZmfJ1gx+y8j3LcMIMRHhXNc3EbvY+Xrtb+w8nF/q9Y1iIvnDee2w2+xMXpzGrsPW9o039tYNY7l/2DnYbDb++c1m9h4pxCBg7IDQuXk8j17cBbvYuWPqGn7LKjr22Rg7557dmL9f1hWb2Ljk5aUcyXP59m2MndFdW/DcFd2x2+x0eWwWhWXOLF3XtzV/vzQJl9tD2798V+5/063nncVDF3YkK89J8pOzyi2/b1h77hzcjn2Z+fR/el655Y+N6sRNvduw5UA2w15YVG75M5cnMb5za1bvOsJlU5eUW/7KNd25sGNzFm85xF0fLfPOtQ7AUMTbN3YmuVUTZq7/jds/WFnu9Z/+vh9nJjZg/oY9/PGT8j+uvr1rIJ2ax/Htz/t5dPov5ZYv/NP5nNkohs9X7uWZmZvKLU99ZCgJsRG8v3Qnr8zfWm75pr+NJNJm5/VF23lnSVqpZXabsO0fFwHw8rwtfJK6p9TyuMgw1j0xAoBnZ27mm5/3l1rerH4kSx8aAsBfv9rAwl8Pllretkksc/54HgAPfr6O1J1HSi1PbhXP9NvPBeDuj1az6bfsUsvPbduIKTf3BeDW91eyK6N0D9fhnZry+oQUAG6YvJzDuaUT8GXdW/DC+G4AXPm/pRSWSXDF3z23x3Dpqz9SVvF3L7fQzWX/Kf/duG9Ye+4c0o7DOUUVLn9sVCduGtCGPUfyGFvB8mcuT2J8r9bl5lclMSYwv1hF5ApgpDHmZu/z64E+xpg7yqw3CZgE0Lp16547d+485X0um/wgO3fuwmN3YOwOPDY7xu4gNjaaBnExeGxhbM8swmN3+NbBZichPoqm9aNwImw8kIuxCcZuw4SF4w4Pp3WzeJol1KPAZVi7+6h1+BbrICYI7ZrE0qx+NDkFbtbuyfKe0xXff52b16dpXBSZeU5+3mN9iW3YEBEEG91bNaBxvSgO5hSybvdR33wRwYaNPm0a0TA2gv2Zhazfk+1dLojYEYTz2jchPjqctEO5rN9X/pfSBR2aEBMRxtb0HDZV8EtoaMemRDrsbPrtKNvSc8stH9nlDOw2Yf3eLHYeLv2PTAQuSmoGWL/y9hzJL7U8zC6M6HwGACt3HuHA0dK/siMdNgaf0xSA5TsyOJRT+ld0TEQY57W3eiku3XaYzLzS/4jrRzno3zYBgMVbDpFdUPpXdKPYCHq3sYZtWrA5nfyi0r9ym8RF0vNMaxikuRsP4HSXPgg0qx9Fcqt4AL7/5TfK/ntp2SCaLi3q4/EYZm04QFltEmLocEY9ilwe5m1KL7e8bZNY2jaJJb/IXe4ACXDOGfVITIghu8DJj1sPl7uVrXPzOFo2iCYzr4hlOzLKvT65ZTxn1I/kUE4hK8scYAF6tG5A43oRpB8tYM3uzHLLeyU2pEFMOPsy81lfwa/wvmc3Ii7Swe6MvHIHaLAO0tHhYew4lMvW9PI/wAa1TyAizM7W9Gx2HMort3zwOU2w24TNv2WzO6P8d29IR+u788u+LPZnlv5uhdmF8zs0AeDnPVmkZ5deHhFmZ0A767uzetcRMsokiOjwMPqd3QiA1LQMqwVWQlyUg16J1ndr2fbD5JZpYcVHh9OjtfXdWrLtULkWVkJsBF1bxgPww5aDuNylv1tN4iJ8LcAFm9Mpe6huHh9FhzPq4fGYCr87rRpG07ZJLE63h8VbD5Vb3qZRDIkJMRQ43Szdfrjc8raNY2nVMJrcQhcr0sp/tzqcUY9m9avmZn8RWWmMSSk3P4AJqh/whDFmhPf5QwDGmH8e7zUpKSkmNTU1IPEopZQKTcdLUIG8+rcCaCcibUQkHLgKmBHA/SmllKpFAnYNyhjjEpE7gO+xLu1MNsaUP0mtlFJKVSCgN7oYY74Fvg3kPpRSStVO2sFfKaVUSNIEpZRSKiRpglJKKRWSNEEppZQKSZqglFJKhaSA3ah7KkTkIHDqQ0lYEoDyt02HPo27+tTEmEHjrm4ad/U50xhTrrBdSCWoqiAiqRXdkRzqNO7qUxNjBo27umncwaen+JRSSoUkTVBKKaVCUm1MUK8HO4BTpHFXn5oYM2jc1U3jDrJadw1KKaVU7VAbW1BKKaVqAU1QSimlQlKtSVAiMlJENovIVhF5MNjxlCQirURkvohsEJFfRORu7/wnRGSviKzxPi4q8ZqHvO9ls4iMCGLsaSLysze+VO+8hiIyW0S2eP828M4XEXnJG/c6EekRpJg7lPhM14jIURG5JxQ/bxGZLCLpIrK+xLyT/nxF5Abv+ltE5IYgxf2ciGzyxjZNROK98xNFJL/E5/5aidf09H6/tnrfm1Swu0DGfNLfieo+1hwn7o9LxJwmImu880Pis64yxpga/8CqN7UNOAsIB9YCnYIdV4n4mgE9vNP1gF+BTsATwP0VrN/J+x4igDbe92YPUuxpQEKZec8CD3qnHwSe8U5fBHwHCNAXWBYCn70d+A04MxQ/b2AQ0ANYf6qfL9AQ2O7928A73SAIcQ8HwrzTz5SIO7HkemW2s9z7XsT73i6s5phP6jsRjGNNRXGXWf5/wGOh9FlX1aO2tKB6A1uNMduNMUXAR8AlQY7Jxxiz3xizyjudDWwEWlTykkuAj4wxhcaYHcBWrPcYKi4B3vVOvwtcWmL+e8byExAvIs2CEF9JQ4BtxpjKRigJ2udtjFkEZFQQz8l8viOA2caYDGPMEWA2MLK64zbGzDLGuLxPfwJaVrYNb+xxxpifjHUEfY9j77XKHeezPp7jfSeq/VhTWdzeVtCVwIeVbaO6P+uqUlsSVAtgd4nne6g8AQSNiCQC3YFl3ll3eE+JTC4+lUNovR8DzBKRlSIyyTuvqTFmv3f6N6CpdzqU4i52FaX/8Yb65w0n//mGWvwAN2H9Si/WRkRWi8hCERnondcCK9ZiwYr7ZL4TofZZDwQOGGO2lJgXyp/1SaktCapGEJFY4HPgHmPMUeC/wNlAN2A/VlM91AwwxvQALgRuF5FBJRd6f42F5L0KIhIOjAE+9c6qCZ93KaH8+R6PiPwFcAFTvLP2A62NMd2BPwJTRSQuWPGVUeO+E2VcTekfYKH8WZ+02pKg9gKtSjxv6Z0XMkTEgZWcphhjvgAwxhwwxriNMR7gDY6dVgqZ92OM2ev9mw5Mw4rxQPGpO+/fdO/qIRO314XAKmPMAagZn7fXyX6+IRO/iEwERgHXepMr3tNkh73TK7Gu4bT3xljyNGC1x30K34lQ+qzDgLHAx8XzQvmzPhW1JUGtANqJSBvvr+argBlBjsnHe574LWCjMeb5EvNLXp+5DCjupTMDuEpEIkSkDdAO6wJntRKRGBGpVzyNdRF8vTe+4p5iNwDTvdMzgAne3mZ9gawSp6qCodSvy1D/vEs42c/3e2C4iDTwnqIa7p1XrURkJPAAMMYYk1difmMRsXunz8L6fLd7Yz8qIn29/0YmcOy9VlfMJ/udCKVjzVBgkzHGd+oulD/rUxLsXhpV9cDq4fQr1i+GvwQ7njKxDcA6TbMOWON9XAS8D/zsnT8DaFbiNX/xvpfNBKm3DVZPpbXexy/FnyvQCJgLbAHmAA298wV41Rv3z0BKED/zGOAwUL/EvJD7vLES6H7AiXVd4Hen8vliXfPZ6n3cGKS4t2Jdnyn+jr/mXfdy7/dnDbAKGF1iOylYSWEb8Are0W2qMeaT/k5U97Gmori9898Bfl9m3ZD4rKvqoUMdKaWUCkm15RSfUkqpWkYTlFJKqZCkCUoppVRI0gSllFIqJGmCUkopFZI0QSkFiMibItIpQNtuLCLLvMPPDKxkvfNF5OsTbKtbyRG3A0Gskd+jA7kPpfyhCUopwBhzszFmQ4A2PwT42RjT3Rjzw2luqxvWfTiBdA+gCUoFnSYoVad4R8f4RkTWish6ERnvnb9ARFJEZEyJWjqbRWSHd3lP7+CbK0Xk+4pGaRerFs8878Cjc0WktYh0wyqfcYl3m1FlXjNSrBpKq7CGrSme31tElnpbXUvEqnEVDjwJjPdua3xF61UQVzMRWeR9zfriVpyIDPe+dpWIfCoisSJyF9AcmC8i86vqc1fqlAT7TmF96KM6H1h32r9R4nl9798FlBn5AvgEuB1wAEuAxt7544HJFWz7K+AG7/RNwJfe6YnAKxWsH4k18kI7rFEiPgG+9i6L41htpaHA5xVt63jrldnPfRwbBcSOVZMsAVgExHjn/5ljNYXSKFMDTB/6CMYj7DTzm1I1zc/A/4nIM1jJoMJTbiLyAJBvjHlVRLoAXYDZ1jBm2LGGnimrH8daQe9jtZwqcw6ww3hLJYjIB0BxSZP6wLsi0g5rmCzHcbbhz3orgMliDVj8pTFmjYich1WU70fvewoHlp4gXqWqlSYoVacYY34Vq1T6RcDfRWSuMebJkuuIyFBgHFYlU7BaN78YY/pVY6h/A+YbYy4Tq4bYglNdzxizSKwyKRcD74jI88ARrCKHVwcgdqWqhF6DUnWKiDQH8owxHwDPYZXSLrn8TKwBWccZY/K9szcDjUWkn3cdh4h0rmDzS7BGtwa4FjhRh4hNQKKInO19XjJZ1OdYOYSJJeZnY52iO9F6Pt73dMAY8wbwJtZ7/gk4V0TaeteJEZH2x9mHUkGhCUrVNUnAchFZAzwO/L3M8olYo4l/6e1U8K2xSntfATwjImuxRoruX8G27wRuFJF1wPXA3ZUFYowpwDql9423k0R6icXPAv8UkdWUPtMxH+hU3EmikvVKOh9Y611nPPCiMeag971+6I13KdYpR4DXgZnaSUIFm45mrpRSKiRpC0oppVRI0gSllFIqJGmCUkopFZI0QSmllApJmqCUUkqFJE1QSimlQpImKKWUUiHp/wFvadQKiAbqygAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "variational_training1 = []\n",
    "variational_training2 = []\n",
    "kernelbased_training = []\n",
    "nn_training = []\n",
    "x_axis = range(0, 2000, 100)\n",
    "\n",
    "for M in x_axis:\n",
    "\n",
    "    var1 = circuit_evals_variational(\n",
    "        n_data=M, n_params=M, n_steps=M, shift_terms=2, split=0.75, batch_size=1\n",
    "    )\n",
    "    variational_training1.append(var1)\n",
    "\n",
    "    var2 = circuit_evals_variational(\n",
    "        n_data=M, n_params=round(np.sqrt(M)), n_steps=M, shift_terms=2, split=0.75, batch_size=1\n",
    "    )\n",
    "    variational_training2.append(var2)\n",
    "\n",
    "    kernel = circuit_evals_kernel(n_data=M, split=0.75)\n",
    "    kernelbased_training.append(kernel)\n",
    "\n",
    "    nn = model_evals_nn(n_data=M, n_params=M, n_steps=M, split=0.75, batch_size=1)\n",
    "    nn_training.append(nn)\n",
    "\n",
    "\n",
    "plt.plot(x_axis, nn_training, linestyle=\"--\", label=\"neural net\")\n",
    "plt.plot(x_axis, variational_training1, label=\"var. circuit (linear param scaling)\")\n",
    "plt.plot(x_axis, variational_training2, label=\"var. circuit (srqt param scaling)\")\n",
    "plt.plot(x_axis, kernelbased_training, label=\"(quantum) kernel\")\n",
    "plt.xlabel(\"size of data set\")\n",
    "plt.ylabel(\"number of evaluations\")\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4b8458a",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "\n",
    "The number of circuit evaluations needed for variational training depends on the number of parameters used. If it is less it performs better than kernel-based methods and vice versa. If variational circuits turn out to be as parameter-hungry as neural networks, kernel-based training will outperform them for common machine learning tasks. However, if variational learning only turns out to require few parameters (or if more efficient training methods are found), variational circuits could in principle match the linear scaling of neural networks trained with backpropagation.\n",
    "\n",
    "The main conclusion that can be drawn is that kernel-based training can perform better when the number of training parameters for variational circuit is significantly less than the number of training samples.\n",
    "\n",
    "With the use of Covalent we can create workflows that can save intermediate results during training and also helps us monitor the training process."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.4 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
